Add wlm resiliency orchestrator (query group service) #15925
Merged
+1,410
−117
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
|
❌ Gradle check result for 139404a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
3 tasks
jainankitk
reviewed
Sep 25, 2024
server/src/main/java/org/opensearch/search/backpressure/SearchBackpressureService.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/wlm/WorkloadManagementSettings.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/wlm/cancellation/QueryGroupTaskCancellationService.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Kaushal Kumar <[email protected]>
|
❌ Gradle check result for 56c3393: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
|
❌ Gradle check result for 3c6adae: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
jainankitk
reviewed
Oct 4, 2024
There was a problem hiding this comment.
@kaushalmahi12 - Looks much better now. Few minor comments, should be ready to merge soon!
server/src/main/java/org/opensearch/wlm/cancellation/QueryGroupTaskCancellationService.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/wlm/stats/QueryGroupStats.java
Outdated
Show resolved
Hide resolved
|
Also, can you merge from main branch to resolve the conflict in |
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
jainankitk
reviewed
Oct 7, 2024
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
|
❕ Gradle check result for 2d7316b: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
jainankitk
approved these changes
Oct 8, 2024
|
The backport to To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-15925-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e24b4c98405d95b781a43fe73ca370c1a1ca452a
# Push it to GitHub
git push --set-upstream origin backport/backport-15925-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.xThen, create a pull request where the |
kaushalmahi12
added a commit
to kaushalmahi12/OpenSearch
that referenced
this pull request
Oct 8, 2024
…ject#15925) * cancellation related Signed-off-by: Kiran Prakash <[email protected]> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <[email protected]> * add better cancellation reason Signed-off-by: Kiran Prakash <[email protected]> * Update DefaultTaskCancellationTests.java Signed-off-by: Kiran Prakash <[email protected]> * refactor Signed-off-by: Kiran Prakash <[email protected]> * refactor Signed-off-by: Kiran Prakash <[email protected]> * Update DefaultTaskCancellation.java Signed-off-by: Kiran Prakash <[email protected]> * Update DefaultTaskCancellation.java Signed-off-by: Kiran Prakash <[email protected]> * Update DefaultTaskCancellation.java Signed-off-by: Kiran Prakash <[email protected]> * Update DefaultTaskSelectionStrategy.java Signed-off-by: Kiran Prakash <[email protected]> * refactor Signed-off-by: Kiran Prakash <[email protected]> * refactor node level threshold Signed-off-by: Kiran Prakash <[email protected]> * use query group task Signed-off-by: Kaushal Kumar <[email protected]> * code clean up and refactorings Signed-off-by: Kaushal Kumar <[email protected]> * add unit tests and fix existing ones Signed-off-by: Kaushal Kumar <[email protected]> * uncomment the test case Signed-off-by: Kaushal Kumar <[email protected]> * update CHANGELOG Signed-off-by: Kaushal Kumar <[email protected]> * fix imports Signed-off-by: Kaushal Kumar <[email protected]> * add queryGroupService Signed-off-by: Kaushal Kumar <[email protected]> * refactor and add UTs for new constructs Signed-off-by: Kaushal Kumar <[email protected]> * fix javadocs Signed-off-by: Kaushal Kumar <[email protected]> * remove code clutter Signed-off-by: Kaushal Kumar <[email protected]> * change annotation version and task selection strategy Signed-off-by: Kaushal Kumar <[email protected]> * rename a util class Signed-off-by: Kaushal Kumar <[email protected]> * remove wrappers from resource type Signed-off-by: Kaushal Kumar <[email protected]> * apply spotless Signed-off-by: Kaushal Kumar <[email protected]> * address comments Signed-off-by: Kaushal Kumar <[email protected]> * add rename changes Signed-off-by: Kaushal Kumar <[email protected]> * address comments Signed-off-by: Kaushal Kumar <[email protected]> * initial changes Signed-off-by: Kaushal Kumar <[email protected]> * refactor changes and logical bug fix Signed-off-by: Kaushal Kumar <[email protected]> * add chanegs Signed-off-by: Kaushal Kumar <[email protected]> * address comments Signed-off-by: Kaushal Kumar <[email protected]> * temp changes Signed-off-by: Kaushal Kumar <[email protected]> * add UTs Signed-off-by: Kaushal Kumar <[email protected]> * add changelog Signed-off-by: Kaushal Kumar <[email protected]> * add task completion listener hook Signed-off-by: Kaushal Kumar <[email protected]> * add remaining pieces to make the feature functional Signed-off-by: Kaushal Kumar <[email protected]> * extend stats and fix bugs Signed-off-by: Kaushal Kumar <[email protected]> * fix bugs and add logic to make SBP work with wlm Signed-off-by: Kaushal Kumar <[email protected]> * address comments Signed-off-by: Kaushal Kumar <[email protected]> * fix bugs and SBP ITs Signed-off-by: Kaushal Kumar <[email protected]> * add missed applyCluster state change Signed-off-by: Kaushal Kumar <[email protected]> * address comments Signed-off-by: Kaushal Kumar <[email protected]> * decouple queryGroupService and cancellationService Signed-off-by: Kaushal Kumar <[email protected]> * replace StateApplier with StateListener interface Signed-off-by: Kaushal Kumar <[email protected]> * fix precommit errors Signed-off-by: Kaushal Kumar <[email protected]> --------- Signed-off-by: Kiran Prakash <[email protected]> Signed-off-by: Kaushal Kumar <[email protected]> Co-authored-by: Kiran Prakash <[email protected]>
jainankitk
pushed a commit
that referenced
this pull request
Oct 8, 2024
* cancellation related * Update CHANGELOG.md * add better cancellation reason * Update DefaultTaskCancellationTests.java * refactor * refactor * Update DefaultTaskCancellation.java * Update DefaultTaskCancellation.java * Update DefaultTaskCancellation.java * Update DefaultTaskSelectionStrategy.java * refactor * refactor node level threshold * use query group task * code clean up and refactorings * add unit tests and fix existing ones * uncomment the test case * update CHANGELOG * fix imports * add queryGroupService * refactor and add UTs for new constructs * fix javadocs * remove code clutter * change annotation version and task selection strategy * rename a util class * remove wrappers from resource type * apply spotless * address comments * add rename changes * address comments * initial changes * refactor changes and logical bug fix * add chanegs * address comments * temp changes * add UTs * add changelog * add task completion listener hook * add remaining pieces to make the feature functional * extend stats and fix bugs * fix bugs and add logic to make SBP work with wlm * address comments * fix bugs and SBP ITs * add missed applyCluster state change * address comments * decouple queryGroupService and cancellationService * replace StateApplier with StateListener interface * fix precommit errors --------- Signed-off-by: Kiran Prakash <[email protected]> Signed-off-by: Kaushal Kumar <[email protected]> Co-authored-by: Kiran Prakash <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This change adds the last piece in WLM resiliency feature (Query Sandboxing). Post this PR we will be raising the ITs for the feature.
Related Issues
Main RFC: #12342
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.